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A PILOT BATING SCALE FOB VORTEX HAZAEU) EVALUATION 


Roger H. Hdh 
Systems Technology, Inc. 


£3UMMARY 

A pilot rating scale for quantification of the hazard associated with 
a wake vortex encounter was developed. In addition to the rating scale, \ 
this report presents the background and rationale utilized in the develop- 
ment of the scale. ®iis rating scale can be used not only to determine a 
measure of haza^, tiut also to provide a means for defining a boundary 
between hazardous and nonhazardous situations. 

The phrases selected for use in the rating scale, as well as the structure 
of the rating scale, were based on the results of a survey of kQ pilots. The 
survey included material on the sonantic properties of certain word combina- 
tions in addition to a choice of format (or structure) for a rating scale. 

INTROEUCTION 

The need for a new rating scale for vortex upsets stems from two facts . 
First, existing handling quality rating scales have very poor selectivity 
at thjp low or unacceptable end. Second, typical vortex encounter^' would 
tend to be rated near( the low end of existing handling quality rating scales. 
Therefore, it seemed unwise to attempt to use these existing scales or even 
to try to modify them to fit our purpose. Instead, the approach taken was 
to define an entirely dew rating scale. As a result of a survey of 48 
pilots, a set of calibrated phrases was obtained in addition to an over- 
whelming preference for the structure-, (or format) of the scale. In this 
survey phrases and two example rating scale structures were presented. 

The two structures were a decision tree scale (akin to the Cooper Harper, 
scale) and a multiple scale rating system. The survey subjects were asked 
to rate the various phrases and to state their preference of struct\ire. 

They were also encouraged to make any ccsnments they considered pertinent. 




R 


BASIC COIRSIBERAXIOirS 

Qualitative measurements made by human observers are basically estimates 
which arise from subjective judgment. These estimates are said to lie along 
a psychological continutim. There are several degrees of sophistication of 
psychophysical scales (for example, see Ref. l). A brief description of the 
four basic levels of psychophysical scales is given baLow. 

• Nominal — This is the lowest level of scale. Its 
function is usually to identify certain objects 
such as numbering of football players or assignment 
of type or modal numbers to classes of objects. 

• Ordinal — Ordinal scales serve to group objects as 
greater than or less than in terms of scrnie represen- 
tative measure. 

• Interval ■ — ‘ Interval scales are simply ordinal scales 
which are linear. That is, the distance between points 
on the scale consist of equal intervals. 

V Ratio — A ratio scale sillows the determination -i of the 
equality of ratios. Absolute temperature is an example 
of a ratio scale. Such a scale implies that an absolute 
2 ;ero exists. 

The primary objective in the present work is to derive a rating scale 
that will allow certain mathematical operations to be performed on the pilot 
rating data. In particular, we would like to average the pilot rating data 
over several pilots as a measure of the overall hazard caused by the wake 
vortex upset. This implies that the scale must be linear, and from the above 
definitions it must therefore be interval as a minimum. The scale may or may 
not be adjectival, that is, we may pick a scale which is simply interval with- 
out words or phrases associated with each of -the numbers indicated along the 
scale. Such scales are termed non-ad jectival and have been shown to be quite 
successful especially for non-experienced raters (see Ref. 3 )* A ccanparison 
of adjectival and non-ad jectival scales for rake vortex upset hazard is shown 
in' Fig. 1 . The main problem with a non -ad jectival scale is that physical 
interpretation of the resulting ratings is almost completely lacking. The 
adjectival scale in Fig. 1 is one step atove a non-ad je<itival scale in that 
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Figure 1 . Ccmparison of_ adjectival and non- adjectival 
interval rating scales. 


at least the end points are labeled. Physical interpretation of the hazard 
with this scale while somewhat better than the non-adjectival scale is still 
less than desirable. For example, it would be difficult to decide ati what 

■j 

point on 'this scale the pilot rating would represent an unacceptableij vortex 
encounter . 

To summarize, the rating scale must be interval (or ratio) to allow 
averaging of the pilot ratings, and must be adjectival to allow physical 
interpretation of the pilot rating in terms of walce_ vortex encounter hazard. 

Adjectival Scales 

The formulation of an adjectival scale involves a basic tradeoff between 
increasing the nimber of word descriptors on the scale for physical interpi^e- 
tation, and maintaining linearity. The adjectives or phrases used to define 
various levels of hazard on the rating scale must have certain basic properties . 
First, they must relate directly to the area Of concern (in our case, the . 
hazard arising fran the wake vortex upset). This is a measure of th e -validity 
of the adjective or phrase. Second, the adjectives or phrases must be 


selective , that is they must be distinguishable by the rater in terras of 
semantic differences in language. The standard deviation of the resulting 
ability of a group of raters to distinguish semantic differences in the 
phrases used on a given scale should be low, and should be approximately 
equal for all phrases. If the adjectives or phrases are to represent even 
increments on an interval scale, they must be distinguishable in equal 
amounts from the case where there is no hazard to the extreme hazard case. 

It is shown in Ref. 2 that this is essentially impossible, with the primary 
area of difficulty occurring at the low or unacceptable end of the scale 
(which happens to be the region of primary interest) . The basic problem 
is that there is insufficient language to provide a constant sensitivity 
scale where the equal intervals are units related to noticeable semantic 
differences. ‘ ■ 

An attempt to quantify a large number of adjectives and phrases in terms 
of their selectivity and sensitivity was made in Ref. 2. A technique called 
the method of successive intervals was used to modify the raw data so as to 
achieve constant standard deviations (sensitivity) and constant intervals 
(selectivity) for each of the adjectives and phrases used in the study. One 
result of that study is shown in Fig. 2. The adjectives shewn on this scale 
all exhibited a standard deviation of approxiraately one rating point ( 
of full scale). With the exception of "bad," the adjectives fall on the 
scale in approximately equal intervals. Unlike the widely used Cooper-Harper 
handling quality rating scale (see Fig. 5); the adjectives in Fig. 2 are not 
assigned integer numerical ratings and instead are allowed to fall at the 
appropriate place along the scale. In this way, the rater has several, 
reference points along the interval scale to relate the physical significance 
of the ratings to the numerical values. Note that there seems to be no word 
to satisfy a numerical rating in the region of ^5. This scale represents a 
compromise between a purely adjectival and a purely interval scale. 

The adjective "uncontrollable" in Fig, 2 does not fall on the interval 
scale as it represents an ultimate value or endpoint which cannot be averaged 
as a scale value. That is, "uncontrollable" could Just as easily be rated a 
15 as an 8. It is therefore treated as a discrete rating (subject checks box) 
to indicate that the ultimate in poor handling qualities has been achieved. 
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Figure 2. An interval rating scale with ' ~ 

adijectival reference points. 

Scaling Concepts 

The pilot opinion rating for a given task is usually the result of the 
consideration of several factors, //T'or exSi^ple, when evaluating'handling 
qualities on the Cooper-Harper scale, the pilot must .consider aircraft char- 
acteristics, demands on the pilot, and adequacy for 'selected task or required 
operation. In deriving a rating scale for evaluating ' bhe hazard due to wake 
vertex upsets we have considered two possible approaches . The first was 
to use a "multiple scale rating system" where the pilot is asked to separately 
rate each element comprising the overall hazard. The second approach was 
to formulate a "decision tree" akin to that used in the Cooper-Harper scale 
(see Fig. 3). 
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» Delinllion of reijuirod operation involves designation cl night phase and/or 
subphases with accompanying conditions. 


Figure 3. Cooper-Harper handling qualities rating scale (Ref. 5)* 



Multiple seals rating system . - Each of the components of a multiple 
scale rating system defines a separate rating boundary. The summation 
of these boundaries will define a composite boundary which accounts for 
all the pertinent factors which combine to define a hazard. This tech- 
nique was used in recently completed automobile handling qualities experi- 
ments, Ref . !<-. The boundaries had the general shape shown in Fig. where 
it is seen that different rating factors define the minimum acceptable 
boundary for various values of the evaluation parameter. With a decision 
tree approach, the rater is forced to define these boundaries himself by 
internally evaluating and wei^ting each of the pertinent factors which 
make up the overall rating. 
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Figure Ij-. Minimum acceptable boundaries derived 
from multiple scale rating system. 


Five tentative scales were initially proposed based, on the assunption 
that the conponents which define a hazard for wake vortex encovmters are 
as follows: 

1. Chances for successful recovery 

2. Aircraft excursions 
5. Subjective cpinion 

4. Donands on pilot 

5. Aircraft control 
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Decision tree rating scale . — The decision tree rating scale is probably 
best exemplified by the well known Cooper -Harper handling qualities scale 
shown in Fig. 3* Here the rater is asked to make Initial yes/no decisions 
which will lead him to one of four categories. Three of these categories 
contain adjectives and phrases which describe aircraft characteristics and 
demands on the pilot. The rater must then decide which of these series of 
adjectives and phrases best describes the current situation. Thus, the 
rater must make the basic tradeoffs between the various factors which make 
up the overall rating. For example, the pilot may feel that deficiencies 
for a particular airplane ''require improvement, " but that the aircraft 
characteristics may be described as "very objectionable but tolerable." 

He is then forced to decide whether to give the airplane a 6 based on the 
..latter or a 7 based on the former characterization. Another problem with 
this type of scale is that the larger number of adjectives and phrases 
which go into making up the final rating reduce the probability of obtain- 
ing a linear scale. However, because of the considerable success enjoyed 
by the Cooper-Harper handling qualities rating scale, it is felt that we 
should at least consider the possibility of using a decision tree type 
scale for the evaluation of wake vortex hazard. 

) 

Pilot Survey 

Adjectives and phrases were form\ilated to indicate various levels of 
hazard in terras of each of the categories discussed in the following section. 
The selectivity and sensitivity of the adjectives and phrases were evaluated 
by a survey of 48 pilots who were asked to locate each phrase on a non- 
adjectival scale of increasing hazard. (See Appendix A.) Since the basic 
purpose of this evaliiiation was to determine the semantic properties of each 
phrase independently, the phrases were placed in random order. A provision 
was eilso made to allow the subjects to grade the suitability of ecch phrase 
on a scale from A to F. Finally, examples of a decision tree scale and a 
multiple rating scale system were presented using identical ssmple phrases 
and adjectives. The subjects were asked to state their preference. 

An example of the survey questionnaire is given in Appendix A. The 
results are discussed in the following section of this report. 
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3ELECTI0K AND VALIMTiaN 07 FINAL SCALE 

; l 

The results of the questionnaire in Appendix A are given in Appendix B J 

where the testi phrases are listed under the following categories. i 

• Chances for Successful Recovery 

■ ‘ [ 

• Aircraft Excursions 

• Subjective Opinion 

• Demands on Pilot 

• Aircraft Control I: 

Each of the phrases included in the questionnaire (Appendix A) were evaluated 

on a " j "Jint non-adjectival scale of increasing hazard. The means and standard i 

s.tlons were calculated to determine the selectivity of each phrase. 

In addition^ the average letter grades were conputed to obtain an estimate 

of the acceptability of each phrase to the pilots as a group (see Appendix B), | 

Those phrases which had means falling at approximately equal intervals and Ij 

which exhibited the lowest standard deviations were selected for use on the 

rating scales. Where two or more phrases were approximately equal, the ji 

best letter grade was the determining factor. :i 

The above "rules" were tempered with a certain amount of judgement in ■: 

order to provide consistency on a given scale. For exanple, on the "aircraft | 

excxirsions" scale, the word excursions appears in the selected phrases for ii 

ratings near 4 and 5 (see Fig. 5)* Therefore, phrase number 1 in Fig. B-5 ’i 

(negligible excursion) was picked over phrase number 2 (similar to light 
txirbulence) even thou^ number 2 had a slightly better average letter grade 
(B- compared to G+) . Likewise, two phrases in the "demands on the pilot" ;■ 

category were shortened in order to enphasize the key adjectives (low, il 

moderate, and extreme) describing the pilot effort. ‘i 

One phrase was given twice in the questionnaire as a brief check on the 
validity of the results (phrases 34 and 52 in Appendix A) . The results are li 

given in Fig. B-2 where it is seen that "aircraft control required moderate jj 

;|- 
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pilot effort" received identical average values (3.0) and letter grades 
(C+) and reasonably close values of standard deviation (0.3l compared to 
0 . 68 ). 


Scale Format 

The multiple scale rating system format was selected over the decision 
tree approach for the following reasons, 

• 37 of 47 pilots (one offered no preference) favored 
the multiple scale rating system format. (See Appendix A 
for scales presented Jor comparison.) 

M Written commentary from pilot respondents indicated that 
most subjects felt that the multiple scale system was 

less confusing to u^a nd would vie ld._results— that-could — ^ 

be-miore-^easilyT.nterpreted . 

• Written coramentary' from several pilot subjects indicated 
dissatisfaction with the decision tree scale because the 
single rating obtained with this scale assumes that 

the pilot agrees with a31 the items leading to any 
final branch . 

Of the ten pilots who selected the decision tree scale on the questionnaire 
(Appendix A), at least two felt that their choice was at least pEirtly due to 
a bias based on extensive use of the Copper-Harper scale (Fig. 3 ). It is 
ejpected that acceptance of the multiple scale rating system by research 
pilots will generally be a matter of overcoming the bias. 

Selectlcai of Phrases 

The category "chances for successful recovery" was found to be unacceptable 
for use on a vortex hazard rating scale. As shown in Appendix E, none of 
these phrases yielded an average rating less than 2.75. ®iis would indicate 
that if recovery is even a question, the hazard level is considered to be 
quite high. Finally, the variability for these phrases was large compared 
to phrases in other categories. ij 

The category "Subjective Opinion" was rejected because of very poor pilot 
acceptance (indicated by an average letter grade of D+). Many of the phrases 
in this category exhibited rather large variability among pilots. 
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"Aircraft Control" was a fairly good category yielding several phrases 
with low variability (a < 0.6) and well separated means. The final rating 
scale is shown in Fig. 5 where the "Aircraft Control" column utilizes phrases 
2, 9, and £5 (from Appendix A) . These phrases yielded an average standard 
deviation* of 0.51 and an average letter grade of B. 

The second category utilized in the final rating scale was "Demands on 
the Pilot." The Pig. 5 phrases (numbers 32; 3^; and 51 ) yielded an rverage 
standard deviation of O.ii-7 and an average letter grade of C+. Only three 
phrases in ..this category (out of 13) had means and standard deviations 
suitable for use on the final scale. However, since the purj.'ose of the 
•phrases is Himp ly-Jn3-provide--anc h o ^^ ^^ p aiTrtB~(5n'~the~T.nterva3r~Bcale. three is 
felt to be an adequate (minimum) number. 

The category "Aircraft Excursions" utilj.zes phrases 23, 2k, 48, and 56 
resulting in an average standard deviation of 0.45 and an average letter 
grade of C+. Unfortunately, the phrases tend to have means that are somewhat 
unequally spaced. However, it is felt that having phrases with low semantic 
variability among pilots (low a) was more in^jortant than equal spacing of 
the phrases on the scale. 


Endpoints aa the Scales - 

Hone of the phrases had means which fell at the extreme ends of' the five 
point hazard scale in Appendix A. Considering that phrases such as "crash 
inevitable..." and "uncontrollable" were included, it was surprising that 
many pilot subjects did not feel that the "ultimate hazard" was defined. 
Likewise, such terras as "aircraft control was not a factor" and "ne^igible 
excursions" did not fall at the extreme low end of the five point hazard 
scale. These results are consistent with previous surveys (Ref. 2) and 
verify the subjective, comparative nature of rating scales (they have no 
associated absolute values). Vfe have allowed for this in the final rating 

*Averaii;e standaj^d de'vic.tion is defined as 

CFavg - j 
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scale (Pig. 5) "by extending the scale below 1 and above 5. Thus, the rater 
can indicate that the appropriate rating does not fall in the range of 1 
to 5 (shovm by putting his mark at less than 1 or greater than 5). 


Nonad^Jectival Hazard Scale 

The phrases which ccai^rise the adjectival rating scales in Fig. 5 are 
located along the scales according to their semantic value (mean) in terms 
of hazard. As with all pilot rating scales, the p/urpose of these phrases 
is to attach some physical significance to the scale values . A final ecale 
which is nonadjectival (no phrases) has been added to provide a direct 

mflapiir-ft-of thp p-ilfit*R o o inion Of the hazard. This non- adjectival scale 

is identical to the scale used to evaluate the phrases in the suarvey. It 
allows correlation of the first three categories (adjectival scales ) with 
the overall hazard without introducing additional semantic properties. 

Experiments wl'to non- adjectival scales (Ref. 5) have shovm good sensitivity 
in discerning changes in the characteristics of systems that are very difficult 
to control, e.g., an vinstable element. It therefore seems very appropriate 
to apply such a scale to wake vortex encounters where control is frequently 
marginal . 

Validation 

Validation of the vortex hazard rating scale can be obtained only through 
its use in research programs where actual vortex encounters are experienced 
and rated. The success or failure of any rating scale is its ability to 
/i produce consistent ratings within and across pilots. A logical measure of 
scale validity might therefore be the standard deviation of ratings across 
pilots for a given vortex encounter situation. ' 

Another measure of validity is a coaiparison of the semantic meaning of 
the phrases utilized in the final scale between research pilots (who will 
be using the scale) and the entire pilot population. It is, of course, 
desirable that the phrases used have the same mean values (same coordinates 
on the hazard scale), similar (low) variability, a, and reasonably good 
pilot acceptance (letter grades) across both pilot groups. A conaparison 
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of the means^ standard deviations, and letter grades for the survey results 
from just research pilots (9) with the results of all the tested pilots (58)* 
is given in Fig. 6, The mean values are seen to be in good agreement indi- 
cating that the phrases used have the same hazard value for research pilots 
(users of the scale) and the total pilot group. Similarly, the standard 
deviations and letter grades are in very close agreement for the two pilot 
groups . A breakdown of the background of the pilots who responded to the 
questionnaire is given in Appendix B. 

qfUESIIONKAIRES 

As discussed under Basic Considerations, questionnaires will be used to 
augment the rating data obtained. This will cover such things as pilot 
background, experience, and the subject's personal interpretation of the 
severity of the vortex encounter. The proposed questionnaire covering pilot 
background and experience is shown in Fig, 7, A tentative questionnaire to 
stimulate pilot commentary required to supplement the rating data is given 
in Fig. 8. It is intended to aid the experimenter in prompting pilot dis- 
cussion after each vortex encounter run; and it may be altered as a result 
of such direct experience. 

CONCLUSION 

A rating scale has been developed to allow evaluation of the hazard 
associated with wake vortex encounters , The format used involves four 
separate ratings, each defining the hazard associated with a specific 
category. Three of the categories involve interval, adjectival scales, 
whereas the fourth scale is a non-ad jectival measure of hazard. The 
adjectives and phrases selected for the scale were based on a survey designed 
to determine the meaning and variability in meaning of descriptive phrases 
across a group of pilots. 


**Results frcm 10 of the respondents were not included in the statistical 
manipulations because of late arrival or misunderstood instructions. 
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Name 


Occupation 


Date_ 


Total Hours (Approximately) 
License and Ratings ' 0 _ 


Age 


Type Equipment Flown: 

□ Military fighter 
I I Heavy aircraft 

□ Light twins 
Q Light singles 

□ Research simulator 

' i r 

Have you ever had a wake vortex encounter? 


If so, hriefly describe. 


/? 


Figure 7 . Pilot background. 


1 6 


J. 



1. Briefly describe the vortex encorunter. 


2. Did you consider the run hazardous? 

f', 


5 . Wbuld you have continued on normal flight path or aborted (e.g,, dis 
continue to climb or institute go-around) if this encounter occurred 
in fli^t? 


U. If the upset was deemed as hazardous, was the primary hazard; 

a) Ground impact? ' j 

b) Structural failure due to vortex? 

fi- 

c) structural failure due to recovery, attempt? 



Figure 8. Questionnaire after each run. 
(Verbal response frcxn subject.) 


I 


I 


I 


1 


I 


APPENDIX A 

VORTEX HAZARD RATHIG SCALE SURVEY qUESTIOMTAIRE 


INSTRUCTIONS 

The purpose of this questionnaire is to evaluate the meanings of various 
words and phrases which could he used to describe the hazard associated with 
a wake vortex encounter. The words and phrases are presented in random order 
on the next few pages. Alongside each word or phrase is a horizontal scale. 
We want you t'^ place an X on the scale in a position that represents your 
impression of the hazard as defined by that particular phrase. 

We are also interested in your overall impression of how useful each 
phrase might be if included in a rating scale. In other words, we want to 
know which phrases you think ought to be included in a rating scale. There- 
fore, please give each phrase a grade (in the space provided beside the 
horizontal scale) using the usual A through F system. That is, an A would 
imply that the phrase would be an excellent one to Include in a rating scale, 
and an F would indicate that you feel the phrase is totally inappropriate for 
use on a vortex hazard rating scale. If you have any comments you would like 
to make, please feel free to mark up the page with whatever comes to mind. 
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1 i Was disoriented — - probability of 

incorrect control application. 


S. Controllable with somewhat inadequate 
precision. 


3 . Similar to moderate turbulence. 


1* . Barely controllable . 


5 . Considerable cause for alarit,. 


6 . Easy to apply correct contre^s. 

7 . Required considerable pilot 
improper use of fli^t controls 
possible . 

8 . Significant loss of control resulting 
in high probability of aborting task. 


9 , Uncontrollable. 


10. Required moderate control to regulate 
against disturbance. 


Disturbance resulted in momentary 
disorientation. 


12. Was concerned about safety of fli^t 
during some portion of the encounter. 


1 
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Increasing Hazard 

Grade 

13. 

Aircraft excuroionB were large enough 

• • 

( 

1 

1 


to cause concern for safety. 

1 2 

3 

4 

s 

'iu. 

Serlousi loss of control. 

i! 

t 1 

1 


1 __ _ 

1 - 2 

3 

4 

5 

13 . 

Marginally controllable. 

1 i 

1 

1 

1 

1 2 

3 

4 

S 

i6. 

Required considerable pilot effort. Fli^t 
control usage for recovery strai^»tforward. 


1 


f .. 


1 2 

3 

4 

5 

17 . 

C- 

Control was completely lost — - feared for 

• * 

1 


1 


ray life. 

1 2 

3 

4 

5 

i3. 

On the verge of losing control. 

1 1 

t 

1 

1 

1 2 

3 

4 

5 

19* 

■ 

Considered it a "close call*" I 

1 

( 1 

1 

. 1 

1 

1 2 

3 

4 

5 

20. 

(i 

Brief period of loss of control- Little 

1 1 

1 . 

_ 1 

1 _ 


effect on fli^t progress- 

1 2 

3 

4 

5 

21. 

High probability of a crash due to large 
excursions . 

.1 ! 

1 

1 

1 


1 2 

3 

4 

5 

22. 

Extensive loss of control. 

, , 


1 

i 

1 2 

3 

4 

S 

CM 

Negligible excursions. 

1 1 

1 

1 

1 

1 2 

3 

4 

S 

Zh. 

Crash inevitable due to large excursions. 

1 1 

1 .. 

1 

1 . 

I 2 

3 

4 

5 
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Increoslng Hozord Grade 


25 . 

Aircraft control was not a factor. 

\ 

1 

1 

1 

1 

1 

Z 

3 

4 

5 

26. 

Slsturbance easily corrected with 

1 


• 

1 

1 


minimum control. 

1 

z 

3 

4 

5 

27. 

No effect on safety 


1 


t 

• 

1 

2 

3 

4 

9 

28. 

Very difficult to control. 

1 ;■ 

U 

/■ 

1 

» 

t. . 

» _ 

1 

1 

2 

3 

4 

5 

29. 

Completely demanding of pilot attention, 

1 

1 


I 

1 


skill, or effort. 

1 

2 

3 

4 

5 

50. 

Similar to li^t turbulence. 

j 

1 

i 

t 

1 

1 

z 

3 

4 

5 

31. 

Required nearly full control to regulate 

1 

1 

1 

1 

1 


against disturbance. 

1 

z 

3 

4 

9 

32. 

Required low pilot effort. 

1 

1 . 

» 

• 


1 

z 

3 

4 

9 

35. 

Feared for my life. 

1 


1 


1 

1 

z 

3 

4 

9 

34. 

Aircraft control required moderate 

1 


1 

t 

1 


pilot effort. 

1 

z 

3 

4 

9 

r. 

35. 

Momentarily lost control — some concern 

1 

1 

1 

_ 1 . 

1 


for safety. 

1 

z 

3 

4 

9 

36 . 

Aircraft lost. 

t 


1 

r 

1 

1 

z 

3 

4 

9 
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12 3 4 5 


I 


49. Demanding of pilot attention^ skill, 
or effort. 


50, Complete loss of control. 


51. Required extrema pilot effort; proper 
control usage difficult and confusing. 


92. Aircraft control required moderate pilot 
effort. 


Slightly more severe disturtance would 
cause concern for saft,by. 


34. A precarious situation. 


55* Largely undemanding of pilot, relaxed. 


56. Noticeable interference with desired 
fli^t path. 


57. Incident not worthy of mention. 


58. Felt I was in serious trouble. 


59* Fair chance for successful recovery 
from upset. 

ii 

i 

i - ■ ■ 

I ' ■ 

1 4 

1 

I; 
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One final item. The following two pages of this questionnaire show 
exan^les of two possible types of rating scales. The first type is called 
a "multiple scale rating system" because it allows the pilot to rate 

) several aspects of the situation separately; and the s^econd type is called 

a "decision tree scale" (for obvious reasons). We would like you to ccai- 
pare these two types of scales, and indicate which you prefer (and give 
reasons if you desire). Bear in mind that the particular phrases shown 
in these example scales are not important, as they will be revised based 
on this questionnaire. What we really want is to get your cpinion regard- 
ing the basic structure — — decision tree vs. multiple scale rating systems. 

I| RATING SCALE PREFERENCE 

I 

‘ My preference for a rating scale structure is*. 


I Multiple Scale Rating System (Figure 1 ) [ 1 

j Decision Tree Scale (Figure 2) I I 

I 

ii Comments : 

I 

!! 

;| 

i 


a 







Pilot 

decisions 


CH^d^CE FOR 



SUCCESSFUL 


AIRCRAFT 

RECOVERY 

CONTHOL 

EXCURSIONS 


Excellent 

Not a factor 

Not a factor 

© 

Good 

Required moderate 
pilot effort 

Similar to moderate 
turbulence 



Some concern 
for safety 


Fair 

Momentarily lost 

Large 


Poor 

Lost for significant 
period of time 

Very large 



© 


0 


■^Extreme 1^ 
concerned 
-for safety^ 


Impossible Completely lost 


Extreme 


© 


Figure A-2. Example of a decision tree scale 










AFFEKDIX B 

RESULTS OF THE SURVEY 

The results of the Appendix A survey are presented in Figs. B-1 through 
B-U. A total of 48 pilots filled out the questionnaires of which 58 were 
used to compute means, standard deviations, and an average letter grade for 
each phrase. Some questionnaires could not “be used for data analysis either 
because they arrived too late to be included in the calculations or because 
the subject mife«;i,.iderstood the instructions. All of the questionnaires were 
utilized to decide on the scale format (decision tree vs. multiple scale 
rating system). A summary of the background of the pilot population 
responding to the questionnaire was 

- 9 research pilots 

- 5 airline pilots 

- 17 FAA pilots 

- 2 military pilots 

- 12 nonprofessional pilots (all with engineering degrees) 

“ 5 flight instructors 

The rating scale format question received the following replies 

- 57 favored the multiple scale rating system 

- 10 favored the decision tree 

- 1 had no preference * 

Five of the nine research pilots favored the decision tree which probably 
reflects their extensive use of the Cooper-Haiper rating scale. 

The general experience level of the respondents was very high with 
24 ATP's (Airline Transport Pilot rating). Of the remaining pilots, 17 had 
a commercial license with an- instrument rating. The average number of hours 
of flying time of all the respondents was 5922. 
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I 


5 


i 


I 


of the 48 respondents indicated that they had experienced one or 
more vortex encounters during their career. One indicated that the encounter 
resulted in a crash. 
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1 . 


2 . 


3 . 

4 . 

3 . 

6 . 


7 . 


8 . 


9 - 


10 . 


11 . 


12 . 


13 . 

14 . 

15 . 


i6. 


17 . 


18. 


Average Rating 

1 - 2345 

Letter I I I I I 

Grade 


Aircraft control was notna factor. 

Controllable with_some7rtaat inadequate 
precision. ' 

Brief period of loss of control. 

Little effect on flight progress. 

Mraientarily lost control — some concern 
for safety. 

Marginally controllable. 

Very difficult to control. 

On the verge of losing control. 

Barely controllable. 

Nearly uncontroUabl.e. 

Significant loss of control resulting 
in hi^ probability of aborting task. 

Lost control for a significant period 
of time — cause for alarm. 

Extensive loss of control. 

Aircraft lost . 

Complete loss of control. 

Serious loss of control. 

Uncontrdlable . 

Complete loss of control — hi^ 
probability of a crash. 

Control was completely lost — feared 
for my life. 



Figure B- 1 , Aircraft control , 
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1 . 

2 . 


3. 


h. 


3. 


6 . 


7. 


8 . 


9. 

10 . 


11 . 


12 . 


15 . 


14. 


15. 


Easy to apply correct controls. 

Disturbance easily corrected with 
minimum control. 

Required low pilot effort. 

Largely undemanding of pilot, relaxed. 

Aircraft control required moderate 
pilot effort. 

Required moderate control to regulate 
against disturbance. 

Aircraft control required moderate 
pilot effort . 

Demanding of pilot attention, skill, 
or effort 

Required considerable pilot effort. 
Flight control usage for recovery 
straightforward. 

Very demanding of pilot attention, 
skill, or effort . 

Required nearly full control to 
regulate against disturbance. 

Was disoriented — hi^ probability of 
incorrect control application. 

Crarpletely demanding of pilot attention, 
skill, or effort. 

Required considerable pilot effort; 
improper use of fli^t controls 
possible. 

Required extreme pilot effort; proper 
control usage difficult and confusing. 


T- 

Letter I 
Grade 


Average Rating 



Figure B-2. Demands on pilot. 
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Average Rating 

231^ 


Letter 

Grade 


1 , Negligible excursions. 

2. Similar to li^t turbulence. 


C+ 

B- 


3 . Noticeable interference with desired 
fli^t path. 


4. Similar to moderate turbulence. 


3 . Aircraft excursions were large enou^ 
* to cause concern for safety. 


D+ 

C+ 

C 


L j Extreme excursions; not possible to 
maintain desired fli^t path. 


B 


7 . Extreme excursions; definitely an 
unsafe condition . 


0. High probability of a crash due to 
large excursions . 


9 , Crash inevitable due to large excursions, 


C+ 

C 


M 




Figure B-3. Aircraft excursions. 
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Letter | 
Grade 


1 . Excellent chance for successful recovery 
from upset. 


C- 


2. Good chance for successful recovery from 
upset. 


3 . Fair chance for successful recovery frcrni 
upset. 


4. Poor chance for successful recovery from 
upset. 


5 . Recovery ir^jossible. 


B- 


"35" 


.70 




.52 


.47 


.36 


.17 


Average Rating 
2 3 

I I I 


1.0 


.63 


.76 


.59 


.76 


Figure B-4. Chances for successful recovery. 
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1 i 


I 


i 


Average Rating 

1 2 5 It ij 
Letter | | | | I 
Grade 


1 . Incident not worthy of mention . 

2. No effect on safety. 

3 . Sli^tly more severe disturbance would 
cause concern for safety, 

4. Disturbance resulted in momentary 
disorientation . 

3. Felt I was in trouble. 

6 . Was concerned about safety of flight 
during some portion of the encounter. 

7. Considered it a "close call." 

8. Considerable cause for alarm. 

9 . A precarious situation, 

10. Felt I was in serious trouble. 

11. Ebctreme concern for safety. 

12. Feared for my life. 



Figure B-5. Subjective opinion. 
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